Papers with NLG datasets
Unveiling the Achilles’ Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have highlighted various neural metrics that align well with human evaluations. |
| Approach: | They propose a black-box adversarial framework that generates strong disagreements between human and victim evaluators. |
| Outcome: | The proposed framework can significantly improve the performance of human and victim evaluators. |
Curate and Generate: A Corpus and Method for Joint Control of Semantics and Style in Neural NLG (P19-1)
Copied to clipboard
| Challenge: | Neural natural language generation (NNLG) models generate syntactically correct utterances from structured inputs without needing hand-crafted rules or templates. |
| Approach: | They propose a method for generating a corpus of parallel meaning representations with rich style markup using freely available and naturally descriptive user reviews. |
| Outcome: | The proposed method can be scalably reused to generate NLG datasets for other domains. |
Contrastive Error Attribution for Finetuned Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for error tracing do not detect faithfulness errors in NLG datasets. |
| Approach: | They propose a framework to identify and remove low-quality training instances that lead to undesirable outputs. |
| Outcome: | The proposed method outperforms existing methods for detecting faithfulness errors in NLG datasets. |